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Abstract 

The classical Grothendieck inequality has applications to the design of approximation al- 
gorithms for NP-hard optimization problems. We show that an algorithmic interpretation may 
also be given for a noncommutative generalization of the Grothendieck inequality due to Pisier 
and Haagerup. Our main result, an efficient rounding procedure for this inequality, leads to a 
constant-factor polynomial time approximation algorithm for an optimization problem which 
generalizes the Cut Norm problem of Frieze and Kannan, and is shown here to have additional 
applications to robust principle component analysis and the orthogonal Procrustes problem. 
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1 Introduction 

In what follows, the standard scalar product on C" is denoted (■,•), i.e, {x,y) = Y4=i ^iYi 
x,y G C". We always think of R" as canonically embedded in C"; in particular the restriction of 
■) to R" is the standard scalar product on K". Given a set S, the space M„(S) stands for all 
the matrices M = {Mij)l-^^ with My G S for all i,] G {1, . . .,n). Thus, M„(M„(R)) is naturally 

identified with the n'^-dimensional space of all 4-tensors M = {Mijid)"^ j, with Mjjki G IR for all 
i,i,k,l G {1, ... ,m}. The set of all n X n orthogonal matrices is denoted 0„ C M,i(K), and the set 
of all n X n unitary matrices is denoted U„ C M„(C). 
Given M = {Mijki) G M„(M„(]R)) denote 



OptR(M) 



def 



sup J2 ^ijklUijVki. 



and similarly, for M = (Myjt/) S M„(M„(C)) denote 



Optc(M)''=^' sup £ MijkiUijVui 

U,VGUn i,j,k,l^l 

Theorem 1. TZ/ere exz'sfs fl polynomial time algorithm that takes as input M G M„(M„(R)) and outputs 
U,V e 0„ such that 

Optj,(M) ^ 0(1) MijkiUijVki. 

Respectively, there exists a polynomial time algorithm that takes as input M G M„(M„(C)) and outputs 
U,V &Un such that 

Optc(M)^0(l) MijuUijWi- 

We will explain the ideas that go into the proof of Theorem [T] later, and it suffices to say at this 
juncture that our algorithm is based on a rounding procedure for semidefinite programs that is 
markedly different from rounding algorithms that have been previously used in the optimization 
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literature, and as such it indicates the availability of techniques that have thus far remained un- 
tapped for the purpose of algorithm design. Prior to explaining the proof of Theorem [T] we list 
below some of its applications as an indication of its usefulness. 

Remark 2. The implied constants in the 0(1) terms of Theorem [T] can be taken to be any number 
greater than Isfl. in the real case, and any number greater than 2 in the complex case. There is 
no reason to believe that the factor 2\/2 in the real case is optimal, but the factor 2 in the complex 
case is sharp in a certain natural sense that will become clear later. The main content of Theorem[l] 
is the availability of a constant factor algorithm rather than the value of the constant itself. In 
particular, the novelty of the applications to combinatorial optimization that are described below 
is the mere existence of a constant-factor approximation algorithm. 



1.1 Applications of Theorem [T] 

We now describe some examples demonstrating the usefulness of Theorem [H The first example 
does not lead to a new result, and is meant to put Theorem [T] in context. All the other examples 
lead to new algorithmic results. Many of the applications below follow from a more versatile 
reformulation of Theorem[T]that is presented in Section |5] (see Proposition |20)) . 



1.1.1 The Grothendieck problem 

The Grothendieck optimization problem takes as input a matrix A G M„ (R) and aims to efficiently 
compute (or estimate) the quantity 

n 

max AijSiSj. (1) 

This problem falls into the framework of Theorem[T]by considering the 4-tensor M G M„ {M„ (R) ) 
given by Majj = Aij and M,y)t; = if either i^jork^l. Indeed, 

n n n 

OptR(M) = max ^ AijUuV,; = max ^ AijXiij, = max ^ Ay£,-^y. 

UyeOn^j^-^ x,i/e[-l,l]"y=i i!,<5e{-l,l}",;y=i 

A constant-factor polynomial time approximation algorithm for the Grothendieck problem 
was designed in IIAN04L where it was also shown that it is NP-hard to approximate this problem 
within a factor less that 1 + Eq for some £o S (0/1)- A simple transformation |AN04II relates 
the Grothendieck problem to the Frieze-Kannan Cut Norm problem liFK99il (this transformation 
can be made to have no loss in the approximation guarantee IIKN121 Sec. 2.1]), and as such the 
constant-factor approximation algorithm for the Grothendieck problem has found a variety of 
applications in combinatorial optimization; see the survey [KN12J for much more on this topic. 
In another direction, based on important work of Tsirelson liTsi87 l, the Grothendieck problem has 
found applications to quantum information theory IICHTW04| . Since the problem of computing 
Optj^(-) contains the Grothendieck problem as a special case. Theorem [T] encompasses all of these 
applications, albeit with the approximation factor being a larger constant. 
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1.1.2 Robust PCA 



The input to the classical principal component analysis (PCA) problem is X, n G N a set of points 
a\, . . . G R". The goal is to find a X-dimensional subspace maximizing the sum of the squared 
I2 norms of the projections of the fl, on the subspace. Equivalently, the problem is to find the 
maximizing vectors in 

N K 

where here, and in what follows, 5ij is the Kronecker delta. This question has a closed-form solu- 
tion in terms of the singular values of the N x n matrix whose f-th row contains the coefficients of 
the point fl,. 

The fact that the quantity appearing in (|2]) is the maximum of the sum of the squared norms of 
the projected points makes it somewhat non-robust to outliers, in the sense that a single long vec- 
tor can have a large effect on the maximum. Several more robust versions of PCA were suggested 
in the literature. One variant, known as "Rl-PCA," is due to Ding, Zhou, He, and Zha [DZHZ06J, 
and aims to maximize the sum of the Euclidean norms of the projected points, namely, 

N , K 1/2 

"^^vE(EI(«"y;)l ) • (3) 

We are not aware of any prior efficient algorithm for this problem that achieves a guaranteed 
approximation factor. Another robust variant of PCA, known as "Ll-PCA", was suggested by 
Kwak LKwaOS], and further studied by McCoy and Tropp IMT12| (see Section 2.7 in [MT12J in 
particular). Here the goal is to maximize the sum of the li norms of the projected points, namely, 

N K 

max Y,1L K^i'i/;)!- (4) 

{yi,yi)^&ij 



yi,...,yKGR" 



In [MT12II a constant factor approximation algorithm for the above problem is obtained for K = 1 
based on [AN04J, and for general K an approximation algorithm with an approximation guarantee 
of O(logn) is obtained based on prior work by So IISoll| . 

In Section 15.11 we show that both of the above robust versions of PCA can be cast as special 
cases of Theorem [H thus yielding constant-factor approximation algorithms for both problems 
and all K G {!,..., n}. 



1.1.3 The orthogonal Procrustes problem 

Let n,d ^ 1 and K ^ 2he integers. Suppose that Si, . . . , Q are n-point subsets of IR"^. The 
goal of the generalized orthogonal Procrustes problem is to rotate each of the Sk separately so as to 
best align them. Formally, write Sj. = {xj, . . . , x^}. The goal is to find K orthogonal matrices 
Ui, . . . , Uk G Od that maximize the quantity 



E EM 

i=l k=l 



(5) 
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If one focuses on a single summand appearing in say J^f^i U]^x\, then it is clear that in order 
to maximize its length one would want to rotate each of the x\ so that they would all point in the 
same direction, i.e., they would all be positive multiples of the same vector. The above problem 
aims to achieve the best possible such alignment (in aggregate) for multiple summands of this 
type. We note that by expanding the squares one sees that Ui, . . . ,Uk ^ 0,^ maximize the quantity 
appearing in ^ if and only if they minimize the quantity ^-L^ E^/=i || UkX^ — LZ/X- 1| 

The term "generalized" was used above because the orthogonal Procrustes problem refers to the 
case K = 2, which has a closed-form solution. The name "Procustes" is a (macabre) reference to 
Greek mythology (see |iittp : //en . wikipedia . org/wiki7Procrustes| ). 

The generalized orthogonal Procrustes problem has been extensively studied since the 1970s, 
initially in the psychometric literature (see, e.g., liCC71 1 lGow75l ITB77I) , and more recent applica- 
tions of it are to areas such as image and shape analysis, market research and biometric identifica- 
tion; see the books IIGD041IDM98H . the lecture notes IISG02ll . and nMP07H for much more informa- 
tion on this topic. 

The generalized orthogonal Procrustes problem is known to be intractable, and it has been in- 
vestigated algorithmically in, e.g., [TB77, Bou82, SB88J. A rigorous analysis of a polynomial-time 
approximation algorithm for this problem appears in the work of Nemirovski llNem07L where 
the generalized orthogonal Procrustes problem is treated as an important special case of a more 
general family of problems called "quadratic optimization under orthogonality constraints", for 
which he obtains a O ( \/n + d + log K) approximation algorithm. This was subsequently improved 
by So lISoTTH to 0(log(n + d + K)).ln SectionlEU we use Theorem[T]to improve the approximation 
guarantee for the generalized orthogonal Procrustes problem as defined above to a constant ap- 
proximation factor. See also Section |5^ for a more complete discussion of variants of this problem 
considered in [Nem07llSoll| and how they compare to our work. 

1.1.4 A Frieze-Kannan decomposition for 4-tensors 

In IIFK99II Frieze and Kannan designed an algorithm which decomposes every (appropriately de- 
fined) "dense" matrix into a sum of a few "cut matrices" plus an error matrix that has small 
cut-norm. We refer to [FK99| and also Section 2.1.2 in the survey [KN121 for a precise formulation 
of this statement, as well as its extension, due to [ AN 04 |, to an algorithm that allows sub-constant 
errors. In Section l53l we apply Theorem [T] to prove the following result, which can be viewed as 
a noncommutative variant of the Frieze-Kannan decomposition. For the purpose of the statement 
below it is convenient to identify the space Mn{M„{C)) of all 4-tensors with M„{C) ® M„(C). 
Also, for M G M„{C) M„(C) we denote from now on its Frobenius (Hilbert-Schmidt) norm by 



def 



n 



M\\2 




Theorem 3. There exists a universal constant c S (0, oo) with the following property. Suppose that 
M G M„(C) (gi M„(C) andO <e^ 1/2, and let 



One can compute in time poly(n, 1/e) a decomposition 

T 

M = Y^oct{At®Bt) + E, (7) 

such that At,Bt G Un, the coefficients af G C satisfy \cit\ = 0(||M||2/w), and Optj^(£) ^ eOpt^(M). 
Moreover, if M & M„(R) ® M„(R) then one can replace Opt£(M) in ^ by Optj^(M), take the coeffi- 
cients ccf to be real, At,Bt S 0„ and E such that Optj^(E) ^ eOptj^(M). 

Theorem |3] contains as a special case its commutative counterpart, as studied in IIFK99llAN04| . 
Here we are given A G M„ (R) with |fl,y | ^ 1 for all i,] G {1, . . . ,n}, and we aim for an error en^. 
Define Mnjj = a^ and Mij]^ = if / 7^ / or A: 7^ /. Then ||M||2 ^ n. An application of Theorem|3](in 
the real case) with £ replaced by £n-^/Optg(M) yields a decomposition 

^ = ELi ^t{atb*t) + E with 

at,bf G [—1,1]" and E G M„(R) satisfying swp^ ^^^_^^Ylfj=i ^ij^i^j ^ Moreover, the number 
of terms is T = 0(l/£^). 

Theorem |3] is proved in Section 15.31 via an iterative application of Theorem [H following the 
"energy decrement" strategy as formulated by Lovasz and Szegedy IILS07| in the context of general 
weak regularity lemmas. Other than being a structural statement of interest in its own right, we 
show in Section 15.31 that Theorem |3] can be used to enhance the constant factor approximation 
of Theorem [1] to a PTAS for computing Optc(M) when Optj;(M) = Q(n||M||2). Specifically, if 
Opt£(M) ^ Kn||M||2 then one can compute a (1 + £)-factor approximation to Opt£(M) in time 
2poly(i/(K£))pQ^y^j^^ This is reminiscent of the Frieze-Kannan algorithmic framework IIFK99| for 
dense graph and matrix problems. 

1.1.5 Quantum XOR games 

As we already noted, the Grothendieck problem (recall Section 11.1.11) also has consequences in 
quantum information theory IICHTW04L and more specifically to bounding the power of entan- 
glement in so-called "XOR games", which are two-player one-round games in which the play- 
ers each answer with a bit and the referee bases her decision on the XOR of the two bits. As 
will be explained in detail in Section 11.21 below, the literature on the Grothendieck problem re- 
lies on a classical inequality of Grothendieck IIGro53| , while our work relies on a more recent yet 
by now classical noncommutative Grothendieck inequality of Pisier |Pis78| (and its sharp form 
due to Haagerup |Haa85ll ). Even more recently, the Grothendieck inequality has been general- 
ized to another setting, that of completely bounded linear maps defined on operator spaces [PS021 
IHM08| . While we do not discuss the operator space Grothendieck inequality here, we remark 
that in IIRV12a| the operator space Grothendieck inequality is proved by reducing it to the Pisier- 
Haagerup noncommutative Grothendieck inequality. Without going into details, we note that this 
reduction is also algorithmic. Combined with our results, it leads to an algorithmic proof of the 
operator space Grothendieck inequality, together with an accompanying rounding procedure. 

In the preprint ||RV12bB written by the last two named authors, the noncommutative and op- 
erator space Grothendieck inequalities are shown to have consequences in a setting that general- 
izes that of classical XOR games, called "quantum XOR games": in such games, the questions to 
the players may be quantum states (and the answers are still a single classical bit). The results 
in |RV12b|l derive an efficient factor-2 approximation algorithm for the maximum success prob- 
ability of players in such a game, in three settings: players sharing an arbitrary quantum state. 
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players sharing a maximally entangled state, and players not sharing any entanglement. Theo- 
rem [T] implies that in all three cases a good strategy for the players, achieving a success that is a 
factor 2 from optimal, may be found in polynomial time. These matters are taken up in IIRV12bB 
and will not be discussed further here. 

1.2 The noncommutative Grothendieck inequality 

The natural semidefinite relaxation of ^ is 

n 

sup sup Yl ^ij^^i'V])' (8) 

deM x,yi={S''-^)" i,j=l 

where S*^^^ is the unit sphere of R''. Since, being a semidefinite program (SDP), the quantity 
appearing in ^ can be computed in polynomial time with arbitrarily good precision (see [GLS93J), 
the fact that the Grothendieck optimization problem admits a constant-factor polynomial time 
approximation algorithm follows from the following inequality, which is a classical inequality of 
Grothendieck of major importance to several mathematical disciplines (see Pisier's survey [Pisl2| 
and the references therein for much more on this topic; the formulation of the inequality as below 
is due to Lindenstrauss and Pelczyriski [LP68| ). 

n n 

sup sup ^ Aij{xi,yj) ^ Kg sup ^ Aye,-^y. (9) 

dGf^ x,i/G(S''-l)« i,j=l £,<5g{-1,1}" y=l 

Here Kq G (0, oo), which is understood to be the infimum over those constants for which ^ holds 
true for all n G N and all A G M„(R), is a universal constant known as the (real) Grothendieck 
constant. Its exact value remains unknown, the best available bounds IRee91l IBMMNlli being 
1.676 < Kq < 1.783. In order to actually find an assignment e, (5 to ([l]l that is within a constant 
factor of the optimum one needs to argue that a proof of ^ can be turned into an efficient rounding 
algorithm; this is done in |AN04| . 

If one wishes to mimic the above algorithmic success of the Grothendieck inequality in the 
context of efficient computation of Optjj(-), the following natural strategy presents itself: one 
should replace real entries of matrices by vectors in £2, i.e., consider elements of Mniii), and 
replace the orthogonality constraints underlying the inclusion U G On, namely, 

n n 

V z, 7 G { 1, . . . , n }, Yj^i^ ^jk = I] ^ki Ukj = Sij, 

k=l k=l 

by the corresponding constraints using scalar product. Specifically, given an n x n vector-valued 
matrix X G M„(4) define two real matrices XX*,X*X G M„(R) by 

V/,;G{1 n}, (XX*)y'i'f](X,-,,Xy,), and (X*X)y "l^' f](X„-,X,y), (10) 

fc=l k^l 

and let the set of d-dimensional vector-valued orthogonal matrices be given by 

On{^^) = {x G M„(R'*) : XX* = X*X = j} . (11) 
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One then considers the following quantity associated to every M G M„(M„(R)), 

SDPK(M)'i^^sup sup £ Mijki {Xij,Yki) . (12) 

Since the constraints that underlie the inclusion X,Y ^ 0„ (R*^) are linear equalities in the pairwise 
scalar products of the entries of X and Y, the quantity SDPr(M) is a semidefinite program and can 
therefore be computed in polynomial time with arbitrarily good precision. One would therefore 
aim to prove the following noncommutative variant of the Grothendieck inequality (HI, 

\/ne'N,\/Me M„{Mn(R)), SDPk(M) ^ 0(1) ■ OptK(M). (13) 

The term "noncommutative" refers here to the fact that Opt^{M) is an optimization problem 
over the noncommutative group On, while the classical Grothendieck inequality addresses an 
optimization problem over the commutative group {—1, 1}". In the same vein, noncommutativity 
is manifested by the fact that the classical Grothendieck inequality corresponds to the special case 
of "diagonal" 4-tensors M G M„(M„(]R)), i.e., those that satisfy Mijki = whenever i ^ j or k ^ I. 

Grothendieck conjectured IIGro53|l the validity of ((13)) in 1953, a conjecture that remained open 
until its 1978 affirmative solution by Pisier l|Pis78| . A simpler, yet still highly nontrivial proof of 
the noncommutative Grothendieck inequality ([13)) was obtained by Kaijser |Kai83[ . In Section H) 
we design a rounding algorithm corresponding to ((13)) based on Kaijser 's approach. This settles 
the case of real 4-tensors of Theorem [H albeit with worse approximation guarantee than the one 
claimed in Remark[2) The algorithm modeled on Kaijser 's proof is interesting in its own right, and 
seems to be versatile and applicable to other problems, such as possible non-bipartite extensions 
of the noncommutative Grothendieck inequality in the spirit of IAMMN06J : we shall not pursue 
this direction here. 

A better approximation guarantee, and arguably an even more striking rounding algorithm, 
arises from the work of Haagerup r HaaSSj j on the complex version of ((13)) . In Section [3] we show 
how the real case of Theorem [T] follows formally from our results on its complex counterpart, so 
from now on we focus our attention on the complex case. 



1.2.1 The complex case 

In what follows we let S^^^ denote the unit sphere of (thus can be identified with the unit 
circle C R^). The classical complex Grothendieck inequality IIGro53l [LP68| asserts that there 
exists Kg (0, oo) such that 



Vn G N, VA G M„(C), sup 



sup 



(14) 



Let denote the infimum over those K G (0, oo) for which ((14)) holds true. The exact value of 
remains unknown, the best available bounds being 1.338 < Kg < 1.4049 (the left inequality is due 
to unpublished work of Davie, and the right one is due to Haagerup l|Haa87|| ). 

For M G M„{Mn{C)) we define 



SDPc(M) = sup sup E MijM{X,j,Yki 



(15) 
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where analogously to (|TT]| we set 



Un{C^) = [x G Mn{C^) : XX* = x*x = ;} 



Here for X e M„(C'*) the complex matrices XX*,X*X G M„{C) are defined exactly as in (|lOll . 
with the scalar product being the complex scalar product. Haagerup proved IIHaa85ll that 

Vn e N, VM G M„(M„(C)), SDPc(M) ^ 2 ■ Optc(M). (16) 

Our main algorithm is an efficient rounding scheme corresponding to inequality ((T6l) . The constant 
2 in (|16)) is sharp, as shown in I|HI95| (see also |Pisl2l Sec. 12]). 

We note that the noncommutative Grothendieck inequality, as it usually appears in the lit- 
erature, involves a slightly more relaxed semidefinite program. In order to describe it, we first 
remark that instead of maximizing over X, Y G U„{C'^) in ((15)) we could equivalently maximize 
over X,y G M„(C'*) satisfying XX*, X*X, YY*, y*y ^ I, which is the same as the requirement 
II XX* II, ||X*X||, II YY* II, II Y*Y II ^ 1, where here and in what follows || ■ || denotes the operator 
norm of matrices. This fact is made formal in Lemma [6] below. By relaxing the constraints to 
1 1 XX* II + ||X*X|| ^2 and ||YY*|| + ||Y*Y|| ^ 2, we obtain the following quantity, which can be 
shown to still be a semidefinite program. 



|M||„<;=^sup sup ^ijki{Xij,Yu) 



rfGN X,y6M„(C'') 

||XX*|| + ||X*X||<2 

llyy. _,_ y*Y||^2 



(17) 



Clearly ||M||„c ^ SDPc(M) forallM G M„(M„(C)). Haagerup proved IHaaSSJ that the following 
stronger inequality holds true for all n G N and M G M„(M„(C)). 

||M|Uc ^2-Optc(M). (18) 

As our main focus is algorithmic, in the following discussion we will establish a rounding algo- 
rithm for the tightest relaxation ([16]| . In Section 1231 we show that the same rounding procedure 
can be used to obtain an algorithmic analogue of ((18)) as well. 



1.2.2 The rounding algorithm 

Our main algorithm is an efficient rounding scheme corresponding to ((T6)) . In order to describe it, 
we first introduce the following notation. Let : K — > R+ be given by 

cp{t) Isechf ^t] = -. (19) 

One computes that Jj^ q){t)dt = 1, so (p is a density of a probability measure }i on R, known as the 
hyperbolic secant distribution. By (JKB95^ Sec. 23.11] we have 

Vfl G (0,oo), / fl'V(Odf = (20) 

It is possible to efficiently sample from fi using standard techniques; see, e.g., IIDev86l Ch. IX.7]. 

In what follows, given X G M„{C'^) and z G we denote by (X,z) G M„{C) the matrix 
whose entries are (X,z)yj. = {Xji,z). 
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Rounding procedure 

1. Let X, y G Mn{C^) be given as input. Choose z G {1, —i^ uniformly at random, and 
sample f G IR according to the hyperbolic secant distribution fi. 

2. Set X, '^=^^ ^(X,z) G M„(C) and Y, -i^ {Y,z) G M„(C). 

3. Output the pair of matrices {A,B) = {A{z,t),B{z,t)) = {Uz\X^f ,V^\Y^\-'^) e U„ x U„ 
where X^ = Jiz | Xz | and Yz = Vz\Yz \ are the polar decompositions of Xz and Yz, respectively. 



Figure 1: The rounding algorithm takes as input a pair of vector-valued matrices X, Y G M„(C'^). 
It outputs two matrices A, B G W„(C). 

Theorem 4. Fix G N and e G (0, 1). Suppose that M G M„(M„(C)) and i/zflf X, Y G U„{C'') are 
such that 

MijM{Xij,YM) ^ (l-£)SDPc(M), (21) 

where SDPc (M) zs ^z'yen zn ((151). T/zen f/ze rounding procedure described in Figure [I] outputs a pair of 
matrices A,B eU„ such that 



E 



£ M,jkiAijB,i ] ^ (l-e)sDPc(M). (22) 



i,j,k,l^l 



Moreover, rounding can be performed in time polynomial in n and log(l/e), and can be derandomized in 
time poly(n, 1/e). 

While the rounding procedure of Figured] and the proof of Theorem H] (contained in Section |2] 
below) appear to be different from Haagerup's original proof of ((T8)) in I HaaSSII , we derived them 
using Haagerup's ideas. One source of difference arises from changes that we introduced in order 
to work with the quantity SDPc(M), while Haagerup's argument treats the quantity ||M||„c. A 
second source of difference is that Haagerup's proof of ((T8|) is rather indirect and nonconstructive, 
while it is crucial to the algorithmic applications that were already mentioned in Section 11.11 for 
us to formulate a polynomial- time rounding procedure. Specifically, Haagerup establishes the 
dual formulation of (|18)) , through a repeated use of duality, and he uses a bootstrapping argument 
that relies on nonconstructive tools from complex analysis. The third step in Figure [T] originates 
from Haagerup's complex-analytic considerations. Readers who are accustomed to semidefinite 
rounding techniques will immediately notice that this step is unusual; we give intuition for it in 
Section [L23] below, focusing for simplicity on applying the rounding procedure to vectors rather 
than matrices (i.e., the more familiar setting of the classical Grothendieck inequality). 

1.2.3 An intuitive description of the rounding procedure in the commutative case 

Consider the effect of the rounding procedure in the commutative case, i.e., when X, Y G M„(C'*) 
are diagonal matrices. Let the diagonals of X,Y he x,y G (C*)", respectively. The first step 
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consists in performing a random projection: for / G {l,...,n} let ay = (^;, z) / \/2 € C and 
fij = {yj,z) I \f2 G C, where z is chosen uniformly at random from {1, — z}" (alternatively, 
with minor modifications to the proof one may choose i.i.d. Zy uniformly from the unit circle, as 
was done by Haagerup l|Haa85L or use standard complex Gaussians). This step results in se- 
quences of complex numbers whose pairwise products a^t/^y, in expectation, exactly reproduce the 
pairwise scalar products (xjt/i/y)- However, in general the resulting complex numbers and j6y 
may have modulus larger than 1. Extending the "sign" rounding performed in, say, the Goemans- 
Williamson algorithm for MAXCUT [ ,GW95| to the complex domain, one could then round each 
aj; and /Sy independently by simply replacing them by their respective complex phase. 

The procedure that we consider differs from this standard practice by taking into account po- 
tential information contained in the modulus of the random complex numbers ajt, /5y. Writing in 
polar coordinates aj. = r^^e'^* and /5y = sye"^i we sample a real t according to a specific distribution 
(the hyperbolic secant distribution and round each a-^ and each /3y to 

4|f e'Xe.+flogr,) g gO ^ ^. def ^i(<f,.-flogs,) g gO ^ 

respectively. Observe that this step performs a correlated rounding: the parameter t is the same for 
all;,/: G {l,...,n}. 

The proof presented in [HaaSS] uses the maximum modulus principle to show the existence of 
a real t for which a/^, by as defined above provide a good assignment. Intuition for the existence 
of such a good t can be given as follows. Varying t along the real line corresponds to rotating the 
phases of the complex numbers ay, at a speed proportional to the logarithm of their modulus: 
elements with very small modulus vary very fast, those with modulus 1 are left unchanged, and 
elements with relatively large modulus are again varied at (logarithmically) increasing speeds. 
This means that the rounding procedure takes into account the fact that an element with modu- 
lus away from 1 is a "miss": that particular element's phase is probably irrelevant, and should 
be changed. However, elements with modulus close to 1 are "good": their phase can be kept 
essentially unchanged. 

We identify a specific distribution }i such that a random t distributed according to }i is good, in 
expectation. This results in a variation on the usual "sign" rounding technique: instead of directly 
keeping the phases obtained in the initial step of random projection, they are synchronously ro- 
tated for a random time t, at speeds depending on the associated moduli, resulting in a provably 
good pair of sequences a^^, bj of complex numbers with modulus 1. 

Roadmap: In Section |2] we prove Theorem H] both as stated in Section [1.2.21 and in a form based 
on ((18]| . The real case as well as a closely related Hermitian case are treated next, first in Section |3] 
as a corollary of Theorem HI and then using an alternative direct rounding procedure in Section HI 
Section|5]presents the applications that were outlined in Section [Lll 

2 Proof of Theorem m 

In this section we prove Theorem HI The rounding procedure described in Figure [T] is analyzed 
in Section I2.1[ while the derandomized version is presented in Section 12.21 The efficiency of the 
procedure is clear; we refer to Section l2!2l for a discussion on how to discretize the choice of t. In 
Section |23] we show how the analysis can be modified to the case of ||M||„c and ((18]| . 
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In what follows, it will be convenient to use the following notation. Given M G M„(M„(C)) 
and X,Y G M„(C'^), define 



(23) 



Thus M(-,-) is a sesquilinear form on M„{C^) x M„(C'^), i.e., M{ccX,^Y) = a|SM(X,y) for all 
X, y G MniC^) and a, j6 G C. Observe that if A, B G M„(C) then 



i,j,k,l=l 



mm 



(24) 



2.1 Analysis of the rounding procedure 

Proof of ((221). Let X,Y e U„ (C^) be vector-valued matrices satisfying ([21]). Let z G {1, -1, /, -iY 
be chosen uniformly at random, and 

X,'i^i=(X,z) and y/=^^-L(y,z) 

be random variables taking values in M„(C) defined as in the second step of the rounding proce- 
dure (see Figured]). Then, 



E4M(x„y,)] = 1e, J^z;zs J2 My„(x,y),(y,,), 



r,s=l 



-M(x,y), 



(25) 



where we used the fact that E [z^Zs ] = for every r,s G {l,...,d}. 
Observe that ((20l) implies that 

Vfl G (0,oo), Ef[fl''] =2a- Et[a^+'^]. 

Applying this identity to the nonzero singular values of Xz (S) Yz, we deduce the matrix equality 



Et {A(S)B) 



Et [ [Uz\Xz 
2Xz(giYz- Et 
lXz®Yz- Et 



Uz\Xz 



Uz\Xz 



\l+it 



\l+it 



Vz\Yz 



|2 + !f 



Kiy.p-'* 



(26) 



where Uz, Vz G Un are such that Xz = Uz\Xz \ and Yz = Vz\Yz \ are the polar decompositions of Xz 
and Yz, respectively (and therefore the polar decomposition of Yz is Yz = Vz\Yz\), and we recall 
that the output of our rounding scheme as described in Figure[T]is A = Uz\Xz\^'^ and B = 14|^z| 
It follows from ^ and ^ that 



Ez,t [ M(A, B) ] = M(X, y) - Ez,t [M{Uz\Xz Vz\Yz\^-") ] . 



(27) 



Our goal from now on is to bound the second, "error" term on the right-hand side of l|27)) . Specif- 
ically, the rest of the proof is devoted to showing that for any fixed i G R we have 



Ez[M{Uz\Xzf^'',Vz\Yz 



\2-it\ 



^ -SDPc(M). 



(28) 
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Once established, the estimate ((28)) completes the proof of the desired expectation bound (|22]l since 

|23a(28) 1 ED /I \ 

E,,f[|M(A,B)|] ^ M(X,y)--SDPc(M) ^ f --ejSDPc(M). 

So, for the rest of the proof, fix some f G K. As a first step towards ((28)) we state the following 
claim. 

Claim 5. Let W G M„ (C^) be a vector-valued matrix, and for every r S {1,. . . ,d} define W,- G M„ (C) hy 
{^r)ij — {^ii)r- Letz G {1, —l,i, —iY be chosen uniformly at random. Writing Wz = {W,z) G M,i(C), 
we have 

d 

Ez [(WzW*)2] = {ww*f + ^ w,(w* w - w;w,)w;, (29) 

i-=i 

Ez [(w^Wz)^] = {w*wf + ^ w;(ww* - w,w;)w,. (30) 

Proof. By definition = ZLi %^r, and recalling ^ we have WW* = E^^^ W,W; and W* W = 
1:^=1 W;W,. Consequently, 

Ez [(W,W*)2] =E,[ X] z^z,z7ZsWpW;W,W; 

= '^vK'^v^v + E (wpw;w,w* + w^w;w,w^ 

p=l M6{l,...,i} 

= Wpw;w,w* + w,w;w,w; - w,w;w,w; 
= ( E Wpw;)' + x: Wp( x: w,*w,) w; - f: Wpw;wpw;, 

p=l p=l p=l 

proving ((29)) . A similar calculation yields ((30)) . □ 
Now, for every i G R define two vector-valued matrices 

F(0,G(0 G M„ (^c^i'-i'''-'>' 

by setting for every G {1, . . . , n} and z G {1, —1, /, —/}'*, 

(f ^ (lIz|X.P+'^) .^^ and (G(0;.)z ^ (v.|r.p--^) . (31) 



Thus, 



M{F{t),G{t)) = ^ Y M(LJz|x,|2+^y,|y,|2-f) =E4M(LJz|x,|2+^y,|y,|2-f)]. (32) 
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Moreover, recalling that = LTzlXzl is the polar decomposition of Xz, we have 



2 



(33) 



Similarly F{t)*F{t) = (X^X^)^ , so that an application of Claim|5]with W = ^X yields, using 
XX* = X*X = I since X G U„{C^), 



Analogously, 



f (f)* + 4 E x.x;x,x; = f (o*f (f) + - E x;x,x;x, = -i. 

r=l r=l 



G{t)G{tr + 1 E WW = G(o*G(o + J E ^; = h- 



(34) 



(35) 



(36) 



The two equations above imply that F{t),G{t) satisfy the norm bounds 

max{\\F{t)F{ty\\, ||f (f) ll. l|G(f)G(f)1|. ||G(f)*G(0|| } ^ ^. 
As shown in Lemma |6] below, ((36)) implies that there exists a pair of vector-valued matrices 

R{t),sit)eUn{c'+^"") 

such that 

M{R{t),S{t)) = M{V2F{t),V2G{t)). (37) 

(This fact can also be derived directly using ((34]) and ((35|) .) Recalling the definition of SDPc(M) 
in ((15)) , it follows that for every f G R, 



1* |M(F(0,G(f))| ^ ^|M(R(f),S(0)| ^ ^SDPc(M), (38) 



□ 



Bz[M{Uz\Xzf+'',Vz\Yzf-'' 
completing the proof of ((28)) . 

Lemma 6. Lei X, Y G M„(C'^) foe smc/z f/zflf max(||X*X||, ||XX* ||, ||y*y||, ||yy* ||) ^1. Then thereexist 
R,S G W„(C''+2"') such that for every M G M„{M„{C)) we have M(R,S) = M(X,y). Moreover, R 
and S can be computed from X and Y in time poly(n, d). 

Proof Let A = ; - XX* and B = 7 - X*X, and note that A, B > and Tr(A) = Tr(B). Write the 
spectral decompositions of A and B as A = A,(m,m*) and B = Y^^i }ij{vjV*) respectively. Set 
a = 11=1 = L;"=i Hj, and define 



R'i'x. 



® 



With this definition we have RR* = XX* + A = 7 and R*R = X*X + B = J, so R G ^/„(C''+2"'). 
Let S ^ Un (C''^^" ) be defined analogously from Y, with the last two blocks of n^ coordinates 
permuted. One checks that M(R, S) = M(X, y), as required. 

Finally, A, B, their spectral decomposition, and the resulting R, S can all be computed in time 
poly(n,d) from X,y. □ 
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2.2 Derandomized rounding 

Note that we can always assume that X,Y ElAn (C"^), where d ^ 2n^. We start by slightly changing 
the projection step. Define X'-, to be the projection Xz = ^(X,z), after we replace all singular 
values of Xz that are smaller than e with e. Then, writing 

2X^ ®Yz =iXz®Yz + i{x[-Xz)®Yz + ix[ ®{Yl- Yz), 

we see that in the analogue of ((27|) the first term is at least M(X, Y) — 4£(iSDPc(M). Here we 
use that ||Xz||, HYzH ^ d which follows by the triangle inequality and X, Y G Un{C^)- For the 
second term in (|27|l . the previous analysis remains unchanged, provided we prove an analogue 
of dSll). Using (O and the analogous equations for F{t)*F{t),G{t)G{t)* and G{tyG{t), it will 
suffice to bound four expressions such as [[Ez [ (Xz(Xz)*)^ ] ||. One checks that the modification to 
the rounding we did can only increase this by (even for each z), hence following the previous 
analysis we get the bound in ^ with SDPc (M) /2 replaced by (1 + e'^)SDPc (M) /2. 

Next, we observe that the coordinates of z need not be independent, and it suffices if they are 
chosen from a four- wise independent distribution. As a result, there are only poly(n) possible 
values of z and they can be enumerated efficiently. Therefore, we can assume that we have a value 
z e {1, —1, z, —iY for which 

\Et[M{U,\X'J',Vz\Y^\-'')] \ ^ (i-e)sDPc(M). (39) 

Notice that for some universal constant c > 0, with probability at least 1 — £, a sample from 
the hyperbolic secant distribution is at most c log(l/e) in absolute value. Therefore, denoting the 
restriction of the hyperbolic secant distribution to the interval [— clog(l/£),clog(l/e)] by and 
using the fact that the expression inside the expectation in l|39)l is never larger than SDPc(M), for 
t' distributed according to }i' we have 

\E,[M{Uz\X',f,Vz\Y^\-''')]\ ^ (l-2e)sDPc(M). 
Moreover, for any f, t' G R, 

\M{Uz\Kf,Vz\Y^\-'') - M{Uz\X',f ,Vz\Y^\- ) I 

= \M{Uz{\x',f-\x',f),Vz\Y',\-'*) +M(LJz|x^|''',yz(|rzr"- l^zr"'))| 

^ |M(LJz(|X^|'*- |X^|'0,l^z|ri|-'*)| + |M(LJz|X;p^Fz(|y;r"-|y;r'0)|- (40) 
The first absolute value in (|40)) is at most 

SDPc(M) • ||L7z(|X^|'*- iX^I'Oll ■ WVzKV'W =SDPc(M)- IIIX^I'*- IX^I'^'II 

^SDPc(M)-log(max{||X^||,||(XO-^||})-|t-t'|. 

We have ||Xz|| ^ d as explained above, and || (XQ^^H ^ 1/e by the way our modified rounding 
procedure was defined. Similar bounds hold for Y^. It therefore suffices to pick t from a grid of 
size 0(log(l/e) max(l/e^,d/e)). 
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2.3 The rounding procedure in the case of (|T8|) 

Theorem |4] addressed the performance of the rounding procedure described in Figure [T] with re- 
spect to inequality ((T6)) . Here we prove that this rounding procedure has the same performance 
with respect to the noncommutative Grothendieck inequaUty ((18)) as well. This is the content of 
the following theorem. 

Theorem 7. Fix n,d e K and e e (0,1). Suppose that M G M„{M„{C)) and that X,Y G M„(C'*) 
satisfy 

max{||XX*|| + ||X*X||,||yy*|| + ||Y*y||} ^ 2, (41) 



and 



X] M,jj,i{Xij,Yu) ^ (l-e)||M||„c, 



(42) 



where ||M||„c '^(^s defined in ||18)) . Then the rounding procedure described in Figure^ outputs a pair of 
matrices A,B ElAn such that 



E 



^ MijkiAijBki 

i,j,k,l=l 



^ ( i -e)||M||„c. 



(43) 



Proof. We shall explain how to modify the argument presented in Section IZTl relying on the nota- 
tion that was introduced there. All we need to do is to replace ((38)) by the assertion 



VfeR, \M{F{t),G{t))\^-\\M\\nc. 



(44) 



To this end we use the following corollary of ClaimjSl which is a slight variant of |Haa85l Lem. 4.1]. 

Corollary 8. Let W G M„(C'*) he a vector-valued matrix and z S {1, —l,i, —iY chosen uniformly at 
random. Writing = (W,z) G M„(C), we have 



||lEz [(W,W*)2] II + ||E, [(W*W,)'] II ^ ( II WW*|| + II W*W|| )'. 
Proof. Starting from ((29)) and noting that 



d_ 



£ w,(w*w - w;w,)w; 



^ \\w*w\\ ■ 



r=l 



\w*w\\ ■ www* 



we obtain the inequality 

||Ez [(w,w;)2]|| ^ ||ww*f + ||w*w|| ■ II WW* II . 

Similarly, from (|30l > we get 

llE^ [{W*W,f] II ^ II W*wf + II WW* II ■ ||W*W|| . 
By summing ((46)) and ((47)) one deduces i 



(45) 



(46) 

(47) 
□ 
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Combining ^ with 05]) forW = X/\/2, we have 



1 .ED 

\\F{t)F{t)*\\ + ||f (0*F(f)|| ^ |(||XX*|| + ||X*X||)2 ^ 1. (48) 

Analogously, 

\\G{t)G{t)*\\ + ||G(f)*G(f)|| ^ |(||^^*|| + 11^*^11)^ ^ 1- (49) 
Recalling the definition of ||M||„c/ it follows from (|48)) and (f49|) that for every f G R, 

E4M(LJ,|X,|2+^y,|y,|2-')]| # |M(f(f),G(0)| ^ ^||M||„,. (50) 
Hence, 

E,,t[\M{A,B)\] ^ M(X,Y)--||M||„, ^ f--ej ||M|U„ 

completing the proof of the desired expectation bound (|43l) . □ 

Remark 9. The following example, due to Haagerup l'Haa851, shows that the factor 2 approx- 
imation guarantee obtained in Theorem [7] is optimal: the best constant in (|T8]l equals 2. Let 
M G M„(Mn(C)) be given by Mijki = 3jk, and Myjc/ = if (z,/) 7^ (1,1). A direct computa- 
tion shows that Optc(M) = 1. Define X,Y e M„(C") by Xy = Yyi = v^2/(n + l) ej G C" 
for j G {1, . . . , n} and all other entries of X and Y vanish (here ei, . . . , e„ is the standard basis of 
C"). Using these two vector-valued matrices one shows that ||M||„c ^ 2n/(n + 1). Recall that in 
the Introduction we mentioned that it was shown in IIHI95| that the best constant in the weaker 
inequality (|16)) is also 2, but the example exhibiting this stronger fact is more involved. 



3 The real and Hermitian cases 



The n X n Hermitian matrices are denoted H„. A 4-tensor M G M„{M„{C)) 
said to be Hermitian if M,-, 



M„(C) ^M„(C) is 



Mjiik for a\\i,i,k,l G {1, . . . , n}. Investigating the noncommutative 
Grothendieck inequality in the setting of Hermitian M is most natural in applications to quantum 
information, while problems in real optimization as described in the Introduction lead to real 
M G Mn{M„{K)). These special cases are treated in this section. 

Consider the following Hermitian analogue of the quantity Opt^ (M) . 



def 



Optc(M) = sup 



A,BeH„ iik,h 
I|A|L||B||<1 



E M 



Note that the convex hull of Un consists of all the matrices A G M„(C) with || A|| ^ 1, so by 
convexity for every M G M„(M„(C)) we have 



Optc(M) = sup E MijkiAjjBki 

A,BeM„{C) i,j,k,l=l 
I|A||,||B||<1 



(51) 
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This explains why Opt^ (M) should indeed be viewed as a Hermitian analogue of Opt^-, (M). The 
real analogue of ((5T|) is that, due to the fact that the convex hull of On consists of all the matrices 
A G M„(K) with ||A|| ^ 1, for every M G M„(M„(R)) we have 

n 

OptR(M) = sup MijkiAjBki . (52) 

A,BgM„(R) i,j,k,l^l 
I|A||,||B|K1 

The following theorem establishes an algorithmic equivalence between the problems of ap- 
proximating either of these two quantities. 

Theorem 10. For every K G [1, oo) the following two assertions are equivalent. 

1. There exists a polynomial time algorithm Alg* that takes as input a Hermitian M G M„(M„(C)) 
and outputs A,B E H„ withmax{\\A\\, \\B\\} ^ 1 flndOpt^(M) ^ K\M{A,B)\. 

2. There exists a polynomial time algorithm Alg that takes as input M G M„{M„{R.)) and outputs 
U,V eO„ such that OptR(M) ^ KM{U, V). 

In Section |3^ we show that for every K > 2\/2. there exists an algorithm Alg* as in part 1) of 
TheoremlTOl Consequently, we obtain the algorithm for computing Opt^{M) whose existence was 
claimed in Theorem [TJ The implication 1) =^ 2) of Theorem [TOl is the only part of Theorem [TOl 
that will be used in this article; the reverse direction 2) ==> 1) is included here for completeness. 
Both directions of Theorem [10] are proved in Section 1331 



3.1 Two-dimensional rounding 

In this section we give an algorithmic version of Krivine's proof IIKri79) that the 2-dimensional 
real Grothendieck constant satisfies Kg (2) ^ v^. The following theorem is implicit in the proof 
of lKri79 . Thm. 1]. 

Theorem 11 (Krivine). Let g -.M ^ IRbe defined by g{x) = sign(cos(x)), and let f : [0, 7z/2) Rbe 
given by 

1 ifO^t^^, 

Extend f to a function defined on all o/R by requiring that it is even and f{x + 7i) = —f{x)for all x G R. 
There exists a sequence {fo2<'+i}^o ^ ^"'-^ that for every L G N f/ie numbers {bo, . . . , fciL+i} can be 
computed in poly(L) time, Yd=L+i |^2£+i| ^ C/Lfor some universal constant C, I^2^+iI = 1' '^^^ 

yx,y G R, cos{x - y) = ^ b2e+i— / f{{2£ + l)x - t)g{t - {2i + l)y)df. 

An explicit formula for the sequence {b2e+i}'^^Q can be extracted as follows from the proof 
of IKri79i Thm. 1]. For any £ ^ 0, define a2i = 0, 

e /(2£ + l)7r\ 16/1 fTT\ 
an^, = (-1) cos(^^)^^,— ^(^-(-1) 
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h = v/2(7r/4)V(3fli), and for i > 0, 

bit+i = E ^rf^2i+i. 

"1 d\{2i+l) 

Then |fl2£+i| = 0{\/ 1^), from which one deduces the crude bound |fc2f+i| = 0(l/£^). 

Two-dimensional rounding procedure 

1. Let £ > Oand,for;,/c G {1, . . . ,n} letxy,yj. G C with \xj\ = |y;f| = 1, be given as input. Let 
f,g, C and {fc2f+i}^o ^ Theorem [TT] 

2. For every k let Qj € [0, 27r) (resp. ^j. G [0, 27r)) be the angle that Xj (resp. j/jt) makes with the 
X-axis. 

3. Select i G [— tt, tt] uniformly at random. Let L = [C/e] and p = 1 — YLT=L+i \ ^2e+i\- Select 
£ G {-1, 0, . . . , L} with probability Pr(-l) = 1 - p and Pr(£) = \b2f.+i | for £ G {0, . . . , L}. 

4. For every j, k,iii^O then set Aj =^ sign{b2e+i) f {{li + - t) and ^k =^ g{t - (2^ + '^)<pk)- 
Otherwise, set Ay = 0, /^jt = 0. 

5. Return {Aj)j^{^ „} and {^k)kG{l n}- 



Figure 2: The two-dimensional rounding algorithm takes as input real 2-dimensional unit vectors. 
It returns real numbers of absolute value at most 1. 

Figure |2] describes a two-dimensional rounding scheme derived from Theorem[TlJ The follow- 
ing claim states its correctness in a way that will be useful for us later. 

Claim 12. Let e > and for every i,k G {!,...,«} let Xj,yk G C satisfy \xj\ = \yk\ = 1. Then 
the rounding procedure described in Figure^runs in time poly(n, 1/e) and returns G R with 

\}ik\ ^ Ifor every i,k G {1, . . ■ ,n}, and 

^[Ajn] = -^3f^(x^-y^)+e(x;,y[), (53) 

where x'j,y[ G L2(IR) are such that ||^y||2/ Hy^lb ^ 1- 

Proof. Fix j,k G {1, . . . ,n} and let 9j,(pk and Ay, be as defined in Steps |2] and |4] of the rounding 
procedure, respectively. Applying Theorem [TTl 



£=0 

where 

def 



00 ^ 

E / f{i2£ + l)ej-t)g{t-{2£ + l)cPk)dt. 
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By definition, cos{9j — (pk) = =K (^yy^)- Using 1 — e ^ p ^ 1, which follows from the bound 
stated in Theorem [TTl //yjt equals 1 — p ^ £ times a weighted average of the product of certain 
values taken by / and g, the former only depending on Oj and the latter on (p^. Equivalently, this 
weighted average can be written as the inner product of two vectors x'j and of norm at most 1. 
Finally, all steps of the rounding procedure can be performed in time polynomial in n and 1 /e. □ 



3.2 Rounding in the Hermitian case 

Let M e M„(M„(C)) be Hermitian, and X, Y e W„(C'*). For every r e {l,...,d} define as usual 

X,, Y, G M„(C) by {Xr)jk = {Xjk)r and (Y,)^^ = {Yj^h Define X', Y' e M^lC^'*) by 

-x: 




and 




G C 



2rf 



Then (X')(X')* = (X')*(X') = (XX* + X*X)/2 = J, so X' G ^^(C^'^) and similarly Y' G ^^(C^''). 
Moreover, since M is Hermitian, |M(X, Y)| = |M(X', Y')|. This shows that for the purpose of 
proving the noncommutative Grothendieck inequality for Hermitian M we may assume without 
loss of generality that the "component matrices" of X, Y are Hermitian themselves. Neverthe- 
less, even in this case the rounding algorithm described in Figured] returns unitary matrices A, B 
that are not necessarily Hermitian. The following simple argument shows how Krivine's two- 
dimensional rounding scheme can be applied on the eigenvalues of A, B to obtain Hermitian ma- 
trices of norm 1, at the loss of a factor \/2 in the approximation. A similar argument, albeit not 
explicitly algorithmic, also appears in |RV12bl Claim 4.7]. 

Theorem 13. Let n be an integer, M E M„ {M„ (C) ) Hermitian, e G (0, 1) and X,Y EUn (C) such that 

|M(X, Y) I ^ (1 -e)SDPc(M). 

Then the rounding procedure described in Figure\3\runs in time polynomial in n and 1/e and outputs a 
pair of Hermitian matrices A! , B' E M„(C) with norm at most 1 such that 



E 



M(A', B'\ 



Vroof. Let A,B E M„(C) be as defined as in Step |2] of Figure |3l and assume as in Figure |3] that 
M(A,B) is real. By Theoremiwe have E[|M(A,B)|] ^ (2 - e) SDPc(M). Hence to conclude it 
will suffice to show that for any fixed pair of matrices A,B E M„(C), 



E 



\M{N,B')\\ ^ ^|M(A,B)| -eSDPc(M), (54) 
-I V2 
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Hermitian rounding procedure 

1. Let X,Y e MniC^) and £ > be given as input. 

2. Let A, B G M„(C) be the unitary matrices returned by the complex rounding procedure 
described in Figure [H If necessary, multiply A by a complex phase to ensure that M(A, B) is 
real. Write the spectral decompositions of A, B as 

A = f^ e'^iUjU* and B = £ e'^'^v^vl, 

i=l k=l 

where dj, (p^ and Uj, E C". 

3. Apply the two-dimensional rounding algorithm from Figure |2] to the vectors Xj e'^' and 
yj, 'M e"t'i<. Let \j, Hi be the results. 

4. Output 

A' Yj ^jUjU* and B' =^ ^ H^kvl- 

/=1 k=l 



Figure 3: The Hermitian rounding algorithm takes as input a pair of vector-valued matrices X, Y G 
M„(C'*). It outputs two Hermitian matrices A' , B' G M„(C) of norm at most 1. 



where A' , B' are as returned by the rounding procedure and the expectation is over the random 
choices made in the two-dimensional rounding step, i.e., Step|3]of Figured Applying Claim [121 



E[|M(A',B')|] ^ X] ^(";"J'^t^^^)E[A;F^" 



V 2 y^;t=l 



E {x'j'yk)MiujU*,Vkvl] 
i,k=l 



= -j=\M{A,B)\-e\M{W,Z)\, 



(55) 



where in the last inequality, for the first term we used that M{ujUp v^v^) is real since M is Hermi- 
tian, and for the second term we defined the vector-valued matrices 



]N = Y, x'j UjU* G M„(L2(C)) and 



def 



Yy[vkvleMniL2iC)). 



One checks that WW* = EJL^ \\x'j\\lujU*. Since ||x^'||2 ^ 1 for all ; G {1,. ..,«}, it follows that 
\\WW*\\ ^ 1. Similarly max{ II W* W||, II ZZ* II, II Z*Z 11} ^ 1. Applying Lemma |6] we obtain R, S G 
UnihiC)) such that M(W,Z) = M(R,S),hence |M(W,Z)| ^ SDPc(M). Eq. dSS]) then implies the 
desired estimate dSH). □ 
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3.3 Proof of Theorem [10] 

In this section we prove Theorem [lOl We first record for future use the following simple lemma, 
which is an algorithmic version of ((52)) . 

Lemma 14. There exists a polynomial time algorithm that takes as input a A-tensor M G M„ (M„ (R) ) and 
two matrices A,B E M„(R) with max{||A||, ||B||} ^ 1 and outputs two orthogonal matrices U,V E On 
such that 

n n 

MijkiAijBki ^ X] MijkiUijVki. 

Proof. Write the singular value decompositions of A, B as A = J^f^i criSif* and B = X]"=i T^igih*, 
where each of the sequences (e,)"^^, {fi)"^^, {gi)"^^, (^;)[Li Q R" is orthonormal, and (7, t G [0, 1]" 
(since max{||A||, ||B||} ^ 1). Now, M(A,B) is given by 

aiTjM{eif*,gjh*). (56) 

Fixing all aj, Zj but, say, a-[, (|56)) is a linear function of a-[, and thus we can shift ai to either —1 or 1, 
without decreasing ((56)) . Proceeding in this way with the other variables (72, ... , a„, Ti, . . . , t„, each 
one in its turn, we obtain £,5 e { — 1, 1}" such that if we define U,V E 0„ by LI = ^Mift) 
and y = LLi ^((gi^n then M(A, B) ^ M(LJ, V), as required. □ 

Proof of Theorem UOl We first prove the implication 1) 2). For any A G M2n(C) define 

Mr Mr M, M e M„(C) through the block decomposition A= ( ^3 ) • Given M G M„(M„(R)) 
let M' G M2„(M2„(]R)) be such that M'(A, B) = M(3fiA2, KB2) for every A, B G H2„. Formally, for 
every k,l G {1, . . . , 2n} we have 



Then M' is Hermitian. Apply the algorithm Alg* (whose existence is the premise of part 1) of 
TheoremllOll to M' to get A, B G H2„ such that Optc(M) ^ X|M(A, B) |. Since A, B G H2„ we have 
A3 = A* and B3 = B|. Because max{|| A ||, ||B||} ^ 1 also max{pA||, pB||} ^ 1. ByLemmalH 
we can therefore efficiently find U,V E On such that 

KM{U,V) ^ K\M{'RA,m)\ = K\M'{A,B)\ ^ Opt* (M') 

= sup |M'(C,D)|^ sup |M'((sO S)^(^o^T))| ^ sup M(S, T) = Optj,(M). 

C,DgH2„ S,TgO„ S,TgO„ 

l|C||,||D||a 

To prove the reverse implication 2) 1), define 1/? : M2n (R) ^ H„ by 

t^iitt)) =l{M + M,+A, + Al)+'-{A2-M + Al-As)eHn. (57) 







-n,k,l — n 


if{i,k) G {1,. 


.,n}2and (/,/) G {n + 1,. 


.,2n}2, 
.,2ny, 
.r2n}\ 


1 




-n,l,k—n 


if {jrl)e{h.. 


.,n}^ and (/,/:) G {n + 1,. 


~ 4 ' 




-n,l,k—n 


if e {1,.. 


. ,n}^ and (;,A:) G {n + 1, . 
.,n}2 and (/,/) G {n + 1,. 




[ 


-n,k,l — n 


if {irk)e{l,. 
otherwise. 


.,2n}\ 
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Suppose that M G M„(M„(C)) is Hermitian and define M" G M2n(M2„(R)) by requiring that 
M"{A,B) = M{xp{A),ip{B)) for every A, B G M2„(]R). Note that M{xp{ A), tp{B)) G R since 
M, ^{B) are all Hermitian. Apply the algorithm Alg of part 2) to M", obtaining two orthog- 

onal matrices U,V e Oin satisfying OptK(M") ^ KM"{U,V) = KM{ip{U),ip{V)). ByLemma[El 
below we have max{ ||t/^(LI)||,||t/^(y)||} ^ 1. Moreover, using Lemma [T6lbelow we have 

Optj,(M")= sup \M{ip{A),xp{B))\^ sup |M (t/. (( i« |f )) ,t/; (( 1^ ))) | • (58) 

A,BeM2„{K) X,YgH„ 
||A||,||B|K1 l|X||,||Y||^l 

Observe that for every X G H„ we have \p [ [ ) ) = ^- Consequently the rightmost term 

in dSHll equals Optc(M). Therefore Opt^fM) ^ K\M{xp{U),xp{V))\, so that the algorithm that 
outputs ^{U), xp{V) has the desired approximation factor. □ 

Lemma 15. Let xp : M2n{^) be given as in ((57)). Then\\xp{Y)\\ ^ \\Y\\ for allY e M2n{^). 

Proof. ForY G M2„(]R) write Z = (y + y*)/2. Setting Z = g where Zi,Z2,Z3,Z4 G M„(R), 

we then have ^{Y) = (Zi + Z^)/! + f(Z2 — Z^)/l. Take z G C" and write z = x + zy, where 
x,y G R". Then 

Hm)z,z)) 

_ {Zix,x) + (Z4y,i/) - {Z2y,x) - {Z^x,y) _^ (Ziy^y) + (Z4X,x) + (Z2X,y) + {Z^y,x) 



2\ \-y J \-y J / 2\ \xj \Xj 

Since i/?(Y) is Hermitian, it follows that ||)/?(y)|| ^ ||Z|| ^ ||y||. □ 
Lemma 16. For et7ery X ^ H„we have || ( 3^ If ) II ~ II ^11 ■ 

Proof. Write Z = ( Ix ) ^ (R) • Since X is Hermitian, Z is symmetric. For every a,b E R", 
a\ fa 



bj'\b}/ = {m)a,a) + {{QX)b,a)-{{QX)a,b) + {{^X)b,b) 
= ^{{X{a-ib),a-ib)). 
Since Z is symmetric and X is Hermitian, it follows that ||Z|| = ||X||. □ 



4 Direct rounding in the real case 

We now describe and analyze a different rounding procedure for the real case of the noncom- 
mutative Grothendieck inequality. The argument is based on the proof of the noncommutative 
Grothendieck inequality due to Kaijser |Kai83 1, which itself has a structure similar to the proof of 
the classical Grothendieck inequality due to Krivine [Kri74 | (see also JamSSJ , the "Notes and Re- 
marks" section of Chapter 1 of | |DJT95[ |, and the survey article |JL01| |), and uses ideas from |Fis78[ 
to extend that proof to the non-commutative setting. 



23 



Real rounding procedure 

1. Let X, y G M„(R'*) be given as input, and let e G { — 1, l}"^ be chosen uniformly at random. 

2. Set Ye =^ (£/ Y). Write the singular value decomposition of Ye as Yg = ^-Lj f,(e) Mj(e)yj(e)*, 
where f/(e) G [0, oo) and M,(£),y,(e) G K". Define 

(^£)t =^ J^mm{ti{e),T}ui{e)vi{e)* , 
1=1 



where t =^ v^/2. 
3. Let X(£) G M„ (R) of norm at most 1 be such that 



M{X{e),{Y,)r)= max \M{X,{Y,)^)\. 

XeM„(K) 

llx||<i 



4. Output the pair A = X(e) and B = ^(Ye)^. 



Figure 4: The real rounding algorithm takes as input X, Y G M„(R.'^). It outputs two real matrices 
A,B e M„(R) of norm at most 1. 

Theorem 17. Groen n G N, M G M„(M„(R)) and rj G (0, 1/2), suppose that X, Y G 0„(]R'*) are swc/z 

I M(X, Y) I ^ (1 - j/)SDPr(M), (59) 

where SDPr(M) is defined in ((12)) . Then the rounding procedure described in Figure^ runs in time 
polynomial in n and outputs a pair of real matrices A,B E M„(]R) with norm at most 1 such that 



E 



M(A, B] 



(1-2'?) 



2 



^ 3^3 SDPk(^)- (60) 

Note that by Lemma [l4l we can also efficiently convert the matrices A, B to orthogonal matrices 
U,V E 0„ without changing the approximation guarantee. 

The proof of TheoremllTlrelies on two claims that are used to bound the error that results from 
the truncation step (step 2) in the rounding procedure in Figure HI The first claim plays the same 
role as Claim|5]did in the complex case. 

Claim 18. Fix X G M„(R.'^) and let e G {—1,1}'* be chosen uniformly at random. Set Xg = (e, X). Then 

'Ee[{X,Xlf] ^ (XX*)2 + 2||X*X||XX*, 
^,[{XlX,f] ^ (X*X)2 + 2||XX*||X*X. 

Proof. Define the symmetric real vector-valued matrix Z by 

def ^ X 

,X* 
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Following the proof of Lemma 1.1 in |Pis78| (which establishes a bound analogous to the one 
proved in Claim|8]for the case of i.i.d. {±1} random variables) and defining as usual Z,- G M„(R) 
by {Zr)ij = {Zij)r for every r G {1, . . . , d}, we have 

d 

E, [(e, Z)4] =Y,4+ E (^'Z,' + ZrZsZrZs + Z.Z^Z,) 

r,sG{l,..,rf} 

EZ')'+ E (Z.Z, + ZsZ,)'. (61) 

r=l r,s£{\,...,d} 

r<s 

Using the inequality [A + B){A + B)* ^ 2{AA* + BB*), which holds for all A,B G M„{1R), we 
can bound the second sum in (|6T]) as follows. 

^ (Z,Z, + ZsZ,)' ^2 (f; Z,( X] Z2)z, + X]Z,( X] Zr)Zs 

r,se{l,...,d} r=l sG{l,...,rf} s=l ^ rG{l d} 

r<s s>r r<s 

=ii:M E 

)-=l ^SG{1 d} 

^2x:z.(x:z^?)z,. 

Replacing Z^ by its definition and using ABA* ^ ||B||AA*, which holds true for every positive 
semidefinite B G M„(R), we arrive at the following matrix inequality: 



- \ lE.[(X*X,)2]j 



xx*)2 \ /^l!x*x|| XX* 

{X*Xf) \ ||XX*||X*X 



The inequality above implies a separate matrix inequality for both diagonal blocks, proving the 
claim. □ 

The second claim appears as Lemma 2.3 in PKai83| (in the complex case). We include a short 
proof for the sake of completeness. 

Claim 19. Let Y G M„(R). for any t > 0, there exists a decomposition Y = + such that 
II YtIIoo ^ T and 

Proof. Define by "truncating" the singular values of Y at t, as is done in step 2 of the rounding 
procedure described in Figure IH so that || Yt-|| ^ t. Define Y^ = Y — Yt-. By definition, Y, Y^ and 
Y^ all have the same singular vectors, and the non-zero singular values of Y^ are of the form }i — t, 
where ^ t is a singular value of Y. Using the bound |/^ — t| ^ }P- / (4t) valid for any ^ ^ t, any 
nonzero eigenvalue A = (f/ — t)^ of Y^(Y^)* (resp. (Y^)*Y^) satisfies A ^ / (4t)^, which proves 
the claim. □ 
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Proof of Theorem \T7\ We shall continue using the notation introduced in Figure IH Claim [18] and 
ClaimlHl Let X, Y G 0„(R'^) satisfying For every e G {-IAY and any t > let 

be the decomposition promised by Claim [l9l Combining the bound from Claim [18] with the one 
from Claim [l9l we see that 



|Ee[yj(yj) 



yy*f + 2||y*y||||yy*||) 



(4t)2' 



(62) 



where the final step of ^ follows from y G 0„ (R''), and the same bound holds on\\Ee [{Y^TY^] 
We have 



|M(X, y)| = |Ee[M(X„ Y,)] \ ^ |E,[M(X,, iYe)r)]\ + |1E^[M(X„ Yj) 

V3, 



^ |E,[M(X„ {Y,)r)] I + ^SDPk(M), 



(63) 



where the last inequality in l|63]) follows from the definition of SDPr(M) and ((62]) . 

To bound the first term in let X, = Zf=iSi{e)wi{£)zi{e)*, {Y,)r = Ef=it'i{e)ui{e)vi{£)*, 
where s,(£),f-(e) ^ and M,(e),w,(e), w,(e),z,(£) G R", be the singular value decompositions of 



Xe, (y£)T/ respectively. Set My(£) =^ M(w,(e)z,(£)*,My(£)z;y(e)*). With these definitions. 



IE, [M(X„ (y,)^)] 



E, 



^ E 



^ E, 



Es,(£)2 Y:M,j{e)t'j{e) 



1/2 



E. 



1/2 



(64) 



1/2 



Note that the rightmost term in ^ is at most (^tE, M{X{£),B{e)) j , where B(£) = |(y£)T 
and X(£) is as defined in the rounding procedure in Figure [4] To bound the leftmost term in (|64)) , 
define =^ £[L;i(r,(£)s;(£)^)a;,(£)z,(£)*, where r;(£) is the sign of My(£)fj(£), so that 

f:s,(£)2|f:M,,(£)f;.(£)l = M(w,,(y,),). 

/=1 ;=1 

Moreover, by definition W^W* = (X^X*)^, so using Claim [18] we have 

||Ee[W;W£]|| = \\Ee [(X;Xe)^] II ^ ||X*Xf + 2||XX*||||X*X|| =3, 
and the same bound holds for ||Ee [W^W*] || . By the definition of {Yi,)^ we also have 

max{||E,[(y,);(y,)^] ||, ||E,[(y,)^(y,);]||} ^t2. 

Hence 



i=l ;=1 



M 



V3' T 



^ v^tSDP]r(M), 



26 



where we used the definition of SDPr(M). Finally, combining ((63)) and ((64)) with the bounds 
shown above we obtain 



(syj ,^ / -,/3 

(1 - /;)SDPk(M) ^ |M(X, y)| ^ v^t^SDPr(M)-E,[M(X(£),B(£))] + ^SDPr(M). (65) 

Setting T = \/3/2 in l(65)l and simplifying leads to the desired bound ((60)l . 

All steps in the rounding procedure can be performed efficiently. The calculation of X(£) in the 
third step of Figure ID can be expressed as a semidefinite program and solved in time polynomial 
in n. Alternatively, one may directly compute X(£) by writing the polar decomposition 

Tt2{M*{I B)) = QP e M„iR), 

where the partial trace is taken with respect to the second tensor, Q is an orthogonal matrix and P 
is positive semidefinite, and taking X(£) = Q*. □ 



5 Some applications 

Before presenting the details of our applications of Theorem [TJ we observe that the problem of 
computing Optj^(M) is a rather versatile optimization problem, perhaps more so than what one 
might initially guess from its definition. The main observation is that by considering matrices 
M which only act non-trivially on certain diagonal blocks of the two variables U, V that appear 
in the definition of Optj^(M), these variables can each be thought of as a sequence of multiple 
matrix variables, possibly of different shapes but all with operator norm at most 1. This allows for 
some flexibility in adapting the noncommutative Grothendieck optimization problem to concrete 
settings, and we explain the transformation in detail next. 

For every n,m ^ 1, let Mm,n (K) be the vector space of real m x n matrices. Given integers k,£ ^ 
1 and sequences of integers (m,), (n,) G N*^, (pj), (qj) E N^, we define BUr (A:, £; (nii), (n,), (pj), {qj)), 
or simply BUr (k, i) when the remaining sequences are clear from context, as the set of all 

k I 

that are linear in both arguments. Concretely, / G Bil^{k,£) if and only if there exists real coeffi- 
cients a.irs,juv such that for every (A,) G etiM^„„,(R) and (Bj) E ^^^^ Mp.,^.{R), 

k £ m; )3; Pj qj 

k}, (By);e{l {}) = E E E E E E ''irs,juv {Ai)rs{Bj)u^. (66) 

!=1 j=l r=l s—1 M=l v—1 

For integers m,n ^ 1, let C Mm,n(R) denote the set of all m x n real matrices U such that 
UU* = J if m ^ n and U*U = I if m ^ n. If m = n then 0„^„ = On is the set of orthogonal 
matrices; 0„ i is the set of all n-dimensional unit vectors; O14 is simply the set { — 1,1}. Given 
/ E BilR(A:, i), consider the quantity 

o^f^{f)^^ sup fm),{Vj)). 

(U;)6®tiO„,„„, 
{Vj)<^®UO,, 
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Note that this definition coincides with the definition of Optj^ (/) given in the introduction when- 
ever / G Bil]R(l, \;n,n,n,n). The proof of the following proposition shows that the new optimiza- 
tion problem still belongs the framework of the noncommutative Grothendieck problem. 

Proposition 20. There exists a polynomial time algorithm that takes as input fc, £ G N, (m,), (n;) G N*^, 
{Pj)'{^j) £ ^^ and f G Bil]R{k,i;{mi),{ni),{pj),{qj)) and outputs (Lf,) G ®\^-^Omi,n, md {Vj) G 
0^=1 Op.,cij such that 

Optj,(/) ^ 0{l)-f{m,{Vj)). 
Moreover, the implied constant in the Oil) term can he taken to he any numher larger than 2a/2. 
Proof. Let G N, (m,), (n,) G [pj), (qj) G be given, and define 

def V—' def \ - def v— ' def v— ' 

m = l^mi, n = n,-, p = ^^pj, q = 2^ qj, 

i=l i=l 7=1 ;=1 

and t =^ max{m,n,p,q}. We first describe how 0f^i Mm;,„;(R) (respectively ©y^i Mp.,q.(R)) can 
be identified with a subset of Mf (R) consisting of block diagonal matrices. For any i G {1, . . . ,k} 
and r G {1, . . . , mi}, s G {1, . . . , n,}, let Fj. ,. G Mf (R) be the matrix that has all entries equal to 
except the entry in position (r + YLj<i^jfS + ILj<i^i)' which equals 1. Similarly, for any G 

{!,...,£} and u G {1, . . . , Pj}, v E {1,. . . ,qj} we let Gi,,i, G Mf (R) be the matrix that has all entries 
equal to except the entry in position (w + J^j^j pi, v + Yai<i which equals 1. Define linear maps 

O : ©ti M„,,,„.(R) ^ Mf(R) and Y : 0^^^ Mp.,g/R) ^ Mf(R) by 

^((^0) = E E E (^O.s and Y((B,)) '^^^^ E E E (S;)".^ 

!=1 r—\ s=l ;'=1 M=l i?=l 

From the definition, one verifies that 

||<I)((AO)|| = max and ||Y((B,-))|| = max ||B,-||. (67) 

lG{l,...,;c} ' /G{1,...,^} 

Let / G Bil]R(fc,^) and {oiirs,]uv) real coefficients as in (|66l)- Define M G Mf(Mf(R)) by 

and M(A, B) =^ if A or B are not in the linear span of the f / , or ci-o respectively. (Recall that 
the notation M{A, B) was introduced at the beginning of Section|2l) With this definition, for any 

{Ai) G M,„„„,(R) and (Bj) G ef^i Mp^,,^(R) we have 

M(<I>((A)),Y((B,))) =/((A,),(B,)). 

We claim that Opt^{f) = Optj^(M), where Optj^(M) is as defined in the Introduction. Indeed, 
by m for any (LZ,) G 0^ O^.n; and {Vj) G ®j^iOp.^. we have \\^{{Ui))\\,\\Y{{Vj))\\ ^ 1, 



28 



hence Opt^{f) ^ Opt^{M). Conversely, let U, V G Ot be arbitrary, and W, V their orthogonal 
projections on lm(0), Im(Y) respectively. Then M{U', V) = M{U, V). Moreover, if 

k e 
(Ul) E M,„,,„,(]R) and (V,') G Mp.,,.{M) 

are such that 0((Lj;)) = LT' and Y((V,')) = ^' thenby dSZll max,g{i^...^^} |m'|| = ||LJ'|| ^ ||LJ|| = 1, 
and similarly maXj^^^ jj 11^/11 ^ 1- As in the proof of Lemma [I4l we may then argue that there 
exists Uj G Omi,ni, yj S Op.^q. such that 

fm),iVj))^fm),{Vl)) = M{U',V')=M{U,V), 

proving the reverse inequality Optj^(/) ^ Opt]jj(M). 

To conclude the proof of Proposition |20] it remains to note that the algorithm of Theorem [H 
when applied to M, produces in polynomial time U,V E Of such that Optj^(M) ^ 0(1) • M(ii, V). 
Arguing as in Lemma [lH (Uj) and (Vj) can be computed from U,V m polynomial time and con- 
stitute the output of the algorithm. □ 



5.1 Constant-factor algorithm for robust PCA problems 

We start with the Rl-PCA problem, as described in Let ai, . . . G IR" be given, and define 
/ G BilK(l, N; (K), (n), (1, . . . , 1), (K, . . . , X)) by 

f{y, (zi zn)) = E E (zou ( E ^/c;(«0;)- 

The condition Z, G Oi^k is equivalent to Z, being a unit vector, while Y G O^,?! is equivalent 
to the K rows of Y being orthonormal. Using that the £2 norm of a vector u G is equal to 
maXj,gsK-i (w, v), the quantity appearing in (|3l) is equal to 

sup /(Y,(Zi,...,Zn)), 

Zi,...,ZnGC'i,k 

which by definition is equal to Opt][^(/). An approximation algorithm for the Rl-PCA problem 
then follows immediately from Proposition |20l The algorithm for Ll-PCA follows similarly. Let- 
ting g G BilR(l, NK; (X), (n), (1, . . . , 1), (1, . . . , 1)) be defined as 

N K n 

g{y,{^xi = E E E M^i)})' 

!=1 k=l j=l 

and using that Z^^ G O14 is equivalent to Z,j; G { — 1, 1}, the quantity appearing in ([1]) is equal to 

sup g(y,(Zi4,...,ZN,K)) = Opt]^(g), 

Zi,i,...,Zn,k£C'i,i 

which again fits into the framework of Propositionl20l 
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5.2 A constant-factor algorithm for the orthogonal Procrustes problem 



The generalized orthogonal Procustes problem was described in Section (|1.1.3|) . Continuing with 
the notation introduced there, let A\, . . . , be d x n real matrices such that the z-th row of Aj^ is 
the vector G IR*^. Our goal is then to efficiently approximate 



P(Ai,...Ak) 



def 



max 

Ui,...UKGOd 



max Y {UkAk, Lf/A;), 



(68) 



' k,l=l 



where we set {A, B) =^ J^f^i YlJ=i ^ij^ij for every two d x n real matrices A, B. We first observe 
that ([68]) is equal to 

K K 

max max ( VUkAk^TViAi) . (69) 



1=1 



It is clear that ((69]) is at least as large as (|68)) . For the other direction, using Cauchy-Schwarz, 



K. K K 

Y^U,A,,Y^ViAj) ^ Y^UkA^ 

k=l 1=1 k=l 



K 

1=1 



so either Ui, . . . , 17^ or Vi, . . . , achieve a value in (|68l) that is at least as large as (l69l ). The desired 
algorithm now follows by noting that (|69)) falls into the framework of Proposition|20l it suffices to 
define / G BilR(X, K; (d, . . . , d), (d, . . . , d), (d, . . . , d), (d, . . . , d) ) by 

/((!Ji,...,LJk),(^i,...,^k)) = (j^UkAk,J^VjAi). 

k=l 1=1 

Finally, from an assignment to (|69|) we can efficiently extract an assignment to (|68)) achieving at 
least as high a value by choosing the one among the {Uj^) or the (V/) that leads to a higher value. 



Comparison with previous work. To compare the approximation guarantee that we obtained 
with the literature, we first note that P{Ai, . . . , Ak) is attained at Ui, . . . ,Uk S Orf if and only if 
Ui,...,Uk are maximizers of the following quantity over all Vi, . . . , G O^- 

E {VkAk,V,Ai) (70) 

k,lG{l,...,K} 

Indeed, the diagonal term (VJcAjt, Vj^Ajt) = ||VjtAjc||2 equals ||Ajt||2, since Vjt is orthogonal. Con- 
sequently, the quantity appearing in jTOl) differs from the quantity defining P(Ai, . . . , A^) by an 
additive term that does not depend on Vi, . . . , Vk- In the same vein, as already mentioned in Sec- 
tion (|1.1.3|) . P(Ai, . . . , Ak) is attained at Ui, . . . ,Uk ^ O^ii and only if Ui, . . . ,Uk are minimizers 
of the following quantity over all Vi, . . . , G O^. 

K 

J2 \\VkAk-ViAi\\l (71) 
k,l=l 

While the optimization problems appearing in (|68)l , ((70)) and ((71]) have the same exact solutions, 
this is no longer the case when it comes to approximation algorithms. To the best of our knowl- 
edge the polynomial time approximability of the minimization of the quantity appearing in ((TT) 
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was not investigated in the literature. Nemirovski |INem07| and So l|Sollll studied the polynomial 
time approximability of the maximization of the quantity appearing in ([70)) : the best known al- 
gorithm for this problem |Soll| has approximation factor 0(log(n + d + K)). This immediately 
translates to the same approximation factor for computing P{Ai, . . . , Aj<c), which was the previ- 
ously best known algorithm for this problem. Our constant-factor approximation algorithm for 
P{Ai, . . . , Ax) yields an approximation for the maximization of the quantity appearing in ((70)) that 
has a better approximation factor than that of [Soil') unless the additive difference Y^^i \\^k\\\^^ 
too large. Precisely, our algorithm becomes applicable in this context as long as this term is at 
most a factor (1 — 1/C) smaller than P{Ai, . . . , A^). This will be the case for typical applications 
in which one may think of each A]^ as obtained from a common A by applying a small perturba- 
tion followed by an arbitrary rotation: in that case it is reasonable to expect the optimal solution 
to satisfy {U^A^, UiAi) ^ \\A\\2 for every Z,fc G {1, . . .,K}) see, e.g., IINem07l Sec. 4.3]. 

5.3 An algorithmic noncommutative dense regularity lemma 

Our goal here is to prove Theorem |3l but before doing so we note that it leads to a PTAS for 
computing Opt£(M) whenever Opt£(M) ^ kh ||M|| 2 for some constant k > (we shall use below 
the notation introduced in the statement of Theorem|3]). 

The idea for this is exactly as in Section 3 of I FK99)I . The main point is that given such an M 
and e G (0, 1) the decomposition in Theorem|3]only involves T = 0(1/ (ke)^) terms, which can be 
computed in polynomial time. Given such a decomposition, we will exhaustively search over a 
suitably discretized set of values at, bt for Tr( AfX) and Tr(BtY) respectively. For each such choice 
of values, verifying whether it is achievable using an X and Y of operator norm at most 1 can be 
cast as a semidefinite program. The final approximation to Opt(2(M) is given by the maximum 
value of I I^^i Ditatht\, among those sequences {at), {bt) that were determined to be feasible. 

In slightly more detail, first note that for any X, Y of operator norm at most 1 the values of 
Tr{AtX) and Tr(Bf Y) lie in the complex disc with radius n. Given our assumption on Opt^ (M), the 
bound on oct stated in Theorem |3] implies \at\ = 0(Opt^(M)/ {xn'^)). Hence an approximation of 
each Tr{AtX) and Tr(Bty) up to additive error skti/T will translate to an additive approximation 
error to M(X,y) of 0(£Opt^(M)). As a result, to obtain a multiplicative (1 ±e) approximation to 
Opt^(M) it will suffice to exhaustively enumerate among (0((n • T/ {exn))'^))^^ possible values 
for the sequences {at), {bt). 

Finally, to decide whether a given sequence of values can be achieved, it suffices to decide two 
independent feasibility problems of the following form: given n x n complex matrices (Af)^^^ of 
norm at most 1 and (flt)t=i ^ C^, does there exist X G M„{C) of norm at most 1 such that 

FKn 

max max{\Re{Tr{AtX) - at)\,\lm{Tr{AtX) - at)\} ^ ? 

tG{l,...,T} T 

This problem can be cast as a semidefinite program, and feasibility can be decided in time that is 
polynomial in n and T. 

Proof of Theorem\3\ The argument is iterative. Assume that {At,Bt}Jl^ Q Un have already been 

def 

constructed. Write Mi = M and 

Mr = M-Y,oit{At®Bt). 
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If Optf^iMj) ^ £Opt^(M) then we may stop the construction. Otherwise, by Theorem [T] and 
multiplying by an appropriate complex phase if necessary, we can find A^-, G U„ such that 

Mr{Ar,Br) ^ " ^) Opt^ (M^, (72) 



.2 2 

with the left-hand side of ([72l ) real. Set 

def Mt(A^,Bt) 



and define M-^+i =^ Mt — oi-t{At: ® Br). By expanding the square we have 

llM f-\\Mf MrjAr^^r? 
\\Mr+l\\2 - \\Mr\\2 - II . ||2||R ||2 

^ 11^-112 -^^^^^Optc(M)^- (73) 

It follows from ((73)) that as long as this process continues, ||M-r||| decreases by an additive term 
of at least e^(l — e)^Opt£(M)^/ (4n^) at each step. This process must therefore terminate after at 
most T ^ 4n^M\\l/{e{l - e)Optc(M))^ steps. 

The "moreover" part of Theorem |3] follows immediately by using the part of Theorem [T] per- 
taining to Opt][^(M) (note that the specific approximation factor does not matter here). □ 

Acknowledgements. We thank Daniel Dadush and Raghu Meka for useful discussions. 
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